Skip to content

BUG: fix concat to work with more iterables (GH8645) #8668

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 29, 2014
Merged

BUG: fix concat to work with more iterables (GH8645) #8668

merged 1 commit into from
Oct 29, 2014

Conversation

waveform80
Copy link
Contributor

closes #8645

Enhances pd.concat to work with any iterable (except specifically undesired ones like pandas objects and strings). A new test is included covering tuples, lists, generator expressions, deques, and custom sequences, and all existing tests still pass. Finally, a "what's new" entry is included (not sure this last is correct - but pull requests guidelines mentioned it)

# like deque and custom iterables (GH8645)
assert_frame_equal(pd.concat((df1, df2), ignore_index=True), expected)
assert_frame_equal(pd.concat([df1, df2], ignore_index=True), expected)
assert_frame_equal(pd.concat((df for df in (df1, df2)), ignore_index=True), expected)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pls add tests for abc.Iterable as well (should work)
just for confirmation (https://docs.python.org/3.3/library/collections.abc.html)
as this can often be a list-like base class

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gr8

one more minor - can u add the issue number in a comment in the tests

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quick clarification: do you mean replace the (GH8645) on line 2212 with (GH8668)? Or should (GH8645) be more prominent (on its own at the top of the function?)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't see that - I like to see the issue right at the beginning of the test function (the actual issue number)
so when someone else looks at this it's easy to find the reason for the test

@jreback jreback added Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff API Design Reshaping Concat, Merge/Join, Stack/Unstack, Explode and removed Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff labels Oct 28, 2014
@jreback jreback added this to the 0.15.1 milestone Oct 28, 2014
@jreback
Copy link
Contributor

jreback commented Oct 28, 2014

@waveform80 minor comment
otherwise excellent pr!

lmk after you add that test and its green

@waveform80
Copy link
Contributor Author

Updated the test to include Iterable (I've pulled it in from collections rather than collections.abc for compatibility with 2.x and 3.2); I also moved the import into the test itself (seems more sensible given it's the only place deque and Iterable are used). Tests still pass on my machine here - hopefully Travis'll confirm the same in a bit!


.. code-block:: python

In [4]: from collections import deque
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also pull out the import and df1,df2 creation before the code block as another ipython block, then u can use it in the 2nd ipython block)

so the code is not duplicated in the display (u can look above how the prior issue does this)

@waveform80
Copy link
Contributor Author

Done and done (common code in "what's new" pulled out into new section above, and bug reference moved to top of the new test)

@jreback
Copy link
Contributor

jreback commented Oct 29, 2014

perfect - ping on green

wish all PRs were this easy to review!

pick some more issues!

@waveform80
Copy link
Contributor Author

ping :)

jreback added a commit that referenced this pull request Oct 29, 2014
BUG: fix concat to work with more iterables (GH8645)
@jreback jreback merged commit eecf160 into pandas-dev:master Oct 29, 2014
@jreback
Copy link
Contributor

jreback commented Oct 29, 2014

see easy peasy :)

thanks for the effort!

@jreback jreback modified the milestones: 0.15.2, 0.15.1 Oct 30, 2014
@waveform80 waveform80 deleted the deque-concat branch October 30, 2014 17:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API Design Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

Successfully merging this pull request may close these issues.

pandas.concat doesn't accept a deque
2 participants